Spatiotemporal Video Upscaling using Motion-Assisted Steering Kernel (MASK) Regression

In this chapter, we present Motion Assisted Steering Kernel (MASK) regression, a novel multi-frame approach for interpolating video data spatially, temporally, or spatiotemporally, and for video noise reduction, including compression artifact removal. The MASK method takes both local spatial orientations and local motion vectors into account and adaptively constructs a suitable filter at every position of interest. Moreover, we present a practical algorithm based on MASK that is both robust and computationally efficient. In order to reduce the computational and memory requirements, we process each frame in a block-by-blockmanner, utilizing a block-based motion model. Instead of estimating the local dominant orientation by singular value decomposition, we estimate the orientations based on a technique similar to vector quantization.We develop a technique to locally adapt the regression order, which allows enhancing the denoising effect in flat areas, while effectively preserving major edges and detail in texture areas. Comparisons between MASK and other state-of-the-art video upscaling methods demonstrate the effectiveness of our approach. Hiroyuki Takeda Electrical Engineering Dept., University of California, 1156 High St. Santa Cruz, CA, 95064, e-mail: [email protected] Peter van Beek Sharp Laboratories of America, 5750 NW Pacific Rim Blvd., Camas, WA, 98607 e-mail: [email protected] Peyman Milanfar Electrical Engineering Dept., University of California, 1156 High St. Santa Cruz, CA, 95064, e-mail: [email protected]